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ABSTRACT 

A  model  for  genetic  algorithms  with  semantic  nets  is  derived  for  which  the 
relationships  between  concepts  Is  depicted  as  a  semantic  net  An  organism 
represents  the  manner  in  which  objects  in  a  scene  arc  attached  to  concepts 
in  the  net.  Predicates  between  object  pairs  are  continuous  valued  truth 
functions  in  the  form  of  an  inverse  exponential  function  (e'^x^).  l:n 
relationships  arc  combined  via  the  fuzzy  OR  (Max  {...)).  Finally, 
predicates  between  pairs  of  concepts  are  resolved  by  taking  the  average  of 
the  combined  predicate  values  of  ihc  objects  attached  to  the  concept  the 
tail  of  the  arc  representing  the  predicate  in  tite  semantic  net.  The  method 
is  illustrated  by  applying  it  to  the  identification  of  oceanic  features  in  the 
North  Atlantic. 

keywords:  genetic  algorithms,  feature  labelling,  semantic  nets,  fitness 
functions 

•  This  work  was  supported  in  part  by  a  grant  from  Naval  Ocean  Research 
and  Development  Acuvily,  Oram  #NOOOI4-89-J-6003 

BACKGROUND 

Gonatic  algorithms  ara  a  problem  solving  method 
requiring  domain-specific  knowledge  that  is  often 
heuristic.  Candidate  aolutions  are  represented  la 
organisms .  Organisma  are  grouped  into  populations 
known  aa  generations  and  ara  combined  in  pairs  to 
produce  subsequent  generations.  An  individual 
organism's  potential  as  a  solution  is  determined 
by  a  fitneaa  function. 

rltneaa  functions  map  organisms  Into  real  numbers 
and  are  used  to  determine  which  organisms  will  be 
used  (and  how  frequently)  to  produce  offspring  for 
the  succeeding  generation.  Fitness  functions 
often  require  heuristic  information  because  a 
precise  measure  of  the  suitability  of  a  given 
organism  (i.e.,  solution)  is  not  always 
attainable.  An  example  is  the  recognition  (i.e., 
labeling)  of  segmerta  in  a  scene.  General 
characteristics  of  objects  in  the  scene  such  as 
curvature,  size,  length,  and  relationship  to  each 
other  may  be  known  only  within  broad  tolerance 
levels.  That  Is,  there  is  great  variability  in 
the  relationships  among  objects  in  different 
scenes .  ^ 

Semantic  nets  (SMs)  are  effective  representations 
of  binary  relationships  between  concepts  (e.g., 
objects  in  a  scene) .  SNs  denote  concepts  via 
nodes  in  a  directed  graph.  The  arcs  are  labelled 


by  predicates.  We  Introduce  here  a  representation 
of  an  organism  whose  fitness  function  evaluation 
la  dependent  upon  an  SN  context . 

Because  relationships  (i.e.,  predicates)  relating 
concepts  are  not  precise,  their  evaluation  is  in 
the  form  of  a  truth  functional  with  range  (0,1] 
rather  than  the  traditional  (0,1).  That  is,  we 
use  fuzzy  logic  (TATS,  7.T9^)  to  combine 
heur is t ical ly  the  information  concerning  a 
part’cular  organism.  Thus,  we  derive  genetic 
algorithms  with  fuzzy  fitness  functions  (GA/F^) . 

GENETIC  ALGORITHMS 

Genetic  algorithms  (GAs)  are  search  procedures 
modelled  after  the  mechanics  of  natural  selection. 
They  differ  from  traditional  search  techniques  in 
several  ways.  First,  GAs  have  the  property  of 
implicit  parallelism,  where  the  algorithm  is 
equivalent  to  a  search  of  the  hyperplanes  of  the 
search  space,  without  directly  testing  hyperplane 
values  [H075,  GO08].  Nearly  optimal  results  have 
been  foun_d  by  examining  as  few  as  one  po*"t  for 
•very  2JO  points  in  the  search  space  [G086] . 
Second,  GAs  ere  randomized  algorithms,  using 
operations  with  ;.ondeterminist ic  results.  The 
results  for  an  operation  depend  on  the  value  of  a 
random  number.  Third,  GAs  operate  on  many 
solutions  simultaneously,  gathering  Information 
from  all  current  points  to  direct  the  search. 
This  factor  mitigates  the  problems  of  local  maxima 
and  noise. 

From  a  mechanistic  view,  genetic  algorithms  are  a 
variation  of  the  generate  and  test  method.  In 
pure  generate  and  test,  solutions  are  generated 
and  sent  to  an  evaluator.  The  evaluator  reports 
whether  the  solution  posed  is  optimal.  In  genetic 
algorithms,  this  generate  and  teat  process  is 
repeated  iteratively  over  a  set  of  aolutions.  The 
•valuator  returns  information  to  guide  the 
selection  of  new  solutions  for  following 
iterations . 

GA  terminology  is  taken  from  genetics.  Each 
candidate  solution  examined  is  termed  an  organism, 
traditionally  represented  as  a  list  The  set  of 
organisms  maintained  is  termed  a  population,  snd 
the  population  at  a  given  time  is  termed  a 
generation.  Each  iteration  envolves  three  steps 
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First  ,  oach  Tr^.imsia  in  the  current  generation  is 
evaluated,  producing  a  numerical  fitness  function 
result.  The  criteria  for  evaluation  is  domain 
specific  information  about  the  relative  merit  of 
that  particular  organism.  Getter  organisms  are 
assigned  higher  fitness  function  values.  Second, 
some  organisms  are  selected  to  form  one  or  more 
organisms  for  the  next  generation.  Specif ica lly , 
the  number  of  copies  of  each  organism  selected  is 
directly  proportional  to  its  fitness  function. 
Third,  some  of  those  organisms  selected  are 
modified  via  genetic  operators.  Each  genetic 
operator  takes  the  chosen  organism(s),  and 
produces  a  new  organism(s).  The  most  common 
genet ic  operators'  include  crossover  and  mutation 
This  iterative  procedure  terminates  when  the 
population  converges  to  a  solution. 

The  crossover  operator  takes  two  organisms 
selected  and  combines  partial  solutions  of  each. 
When  organisms  are  represented  with  lists,  single 
point  crossover  can  be  viewed  as  combining  the 
teft  hand  side  of  one  organism  chosen  with  the 
right  hand  side  of  the  other,  and  conversely. 
This  creates  two  offspring.  The  crossover  point, 
‘•hat  point  where  the  crossover  takes  place,  is 
randomly  determined. 

The  mutation  operator  us*»s  a  minimal  change 
strategy.  It  takes  a  selected  organism,  and 
changes  the  va 1 ue  at  one  randomly  determined 
position  This  corresponds  to  a  tight  local 
search.  The  offspring  produced  is  identical  to 
the  parent  except  at  the  mutation  point. 


GENETIC  ALGORITHM  PROBLEM  MODEL 
TOR  OCEANIC  FEATURE  LABELING 

Scene  recognition  is  an  application  for  which  the 
GA  model  we  propose  is  suited.  For  example,  Fig 
1(a)  is  a  segmented  image  of  the  North  Atlantic 
for  which  Fig,  1(b)  is  the  original  image.  The 
lines  (referred  to  here  as  segments,  s^,  S2»  ) 

represent  boundaries  between  warm  and  cold  regions 
of  sea  water.  The  problem  is  to  classify  the 
segments  as  Gulf  Stream  North  Wall  (MW) ,  Gulf 
Stream  South  Wall  (SW)  ,  cold  eddies  (CE) ,  warm 
•ddiea  (WE)  ,  continental  shelf  (CS)  ,  and  "other" 
(0)  . 

Relationships  which  can  be  expressed  as  fuzzy 
truth  functions  are  known  to  ex*  st  within  or 
between  c lassif icat ions .  Principal  among  these 
are  (1)  the  average  width  of  the  Gulf  Stream  is  50 
kilometers,  (2)  the  average  diameter  of  an  eddy  is 
100  kilometers,  (3)  cold  eddies  are  usually  south 
of  the  Gulf  Stream,  and  (4)  warm  eddies  are 
usually  north  of  the  Gulf  Stream.  To  these  one 
must  add  the  trivial  (yet  necessary)  relationships 
such  as  the  south  wall  is  at  a  lower  latitude  than 
the  north  well  and  the  known  geophysical 
coordinates  of  continental  shelves. 

A  scene  consisting  of  classif icat ion  categories 
(catx,  c  a t  2  *  catn)  and  r 0 1 a t i ° n 3 h ip * 

expressed  as  truth  functions  P^^j,  ...) 

between  categories  can  be  modelled  as  a  semantic 
net  (or,  more  precisely,  an  association  list) .  A 
generic  one  is  shown  in  Fig.  2.  Segments  are 
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Segmented  Image 


i.  Original  Infrared  Image 


Figure  l.  Oceanic  Features  (North  Atlantic) 


Figure  2.  Generic  Semantic  Net  for  Oceanic  Features 


attached  to  the  categories  via  the  INST  (instance) 
relation.  An  allele  (or  gene)  is  a  category  name. 
An  organism  is  a  list  of  categories,  one  allele 
for  each  segment.  For  example,  given  six  segments 
then  <NW,  NW,  SW,  cs,  CE,0)  and  (CE,  SW,  CE,  0,  0, 
CS)  are  representative  organisms.  Formally,  let 
an  association  list  be  defined  as  A  =»  <V,  P>  where 
V  =  lcat1(  cat2,  c  a  t  m )  is  a  sat  of 
categories,  and  P  *  (Ppj(g)  I  j  <  »,  g  * 
1,2,...,  rpj)  is  a  set  of  binary  predicates. 

These  predicates  describe  the  relationships 
between  categories  and  the  ideal  relationship 
between  segments  assigned  to  these  categories. 

Let  an  organism  for  spatial  labeling  is  defined  as 
Q  =  <S,  INST>,  where  S  =  (Sp,  s2,  .  ..,  sn)  is  a 
set  of  segments,  and  INST:  S  ->  V  is  a  function. 

Crossover  Operators 

There  are  three  applicable  crossover  operators. 
These  include  single  point  crossover,  two  point 
crossover,  and  varying  multiple  point  crossover 
(BOS7]  .  Crossover  operators  require  the 
imposition  of  a  total  order  on  the  segments  in  S. 
Let  Sp  <  Sj  if  1  <  j;  s^wSjifiwj;  Sp  >  8j  if 
1  >  j.  Denote  by  INST0p  the  instance  mapping  for 
organism  Op . 

Single  Point  Crossover.  Given  <  Sp,  Sj,  ...,  an>, 
choose  a  random  integer  k,  1  <  k  <  n.  For  parent 
organisms  01  and  02  create  an  offspring.  O',  such 
that 

(  INST01  ( s  p )  if  i  S  k 
INST0,  (8l)  -  { 

\  INSTo2  (Sp)  if  i  >  k 

Two  Point  Crossover.  Let  <  Cp  e2,  ..,  an>  be  a 
circular  list.  Formally,  aucc(Sp)  »  *i+l 
(pred(s1+1)  ■  Sp)  if  i  <  n  and  succ(sn)  »  aj 
(pred(Sp)  ■  sn) .  Choose  two  random  integers,  kl 
•nd  k2 .  For  parent  organisms  Op  and  02  create  an 
offspring,  0'  such  that 


(  INST01  (Sp)  if  Sp  t  (Skl,  succ(skl), 
INSTq,  (sp)  =  ^  ...,  pred  (sy.2  >  > 

^  INSTo2  (Sp)  otherwise 

Varying  Multiple  Point  Crossover.  For  parent 
organisms  Op  and  02,  create  an  offspring  0'  such 
that 

(  INST01  (sp)  with  probability  0.5 

INSTq,  (Sp)  -  f 

^  INST02  (Sp)  with  probability  0.5 


Mutation  Operator 

Our  mutation  operator  selects  one  segme-.t  randomly 
and  assigns  it  to  a  randomly  determined  category. 
Choose  two  random  integers  kl,  1  <  kl  <  n,  and  k2. 

1  <  k2  <  m.  Remove  skp  from  its  current  category 
in  organism  O  and  att»ch  it  to  catk2  (l.e.,  set 
INSTq  (skl)  -  cat^)  . 

Fitness  Function 

For  the  model,  the  fitness  function  is  the  sum  of 
all  satisfied  predicates  in  the  semantic  net.  Let 
E  denote  the  function.  Let  Pp^^,  be  defined  as 
above,  with  m  possible  categories .  Then 


E  -  Z  I  I  Fp4(9>  <*> 

j-t  1»1  g-l  i 

Pp-l?)  is  a  predicate  for  a  relationship  between 
;stegories,  i  and  j.  Each  predicate  Pp,^  has  a 
corresponding  derived  predl. ite,  pradpjW  (k.  1). 
for  an  analogous  relationship  between  segments  ek 
and  Sp,  where  sk  is  in  category  1  and  Sp  is  in 
category  J.  PpjW  is  interpreted  based  on  the 
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normalized  truth  value  of  the  derived  predicate.  Such  relationships  can  be  considered  as  ordinary 

Specifically,  propc sitional  truth  values. 


(  Z  Z  (k,  1) 

I  _Si_Sk _  (2) 

|  Icatjl  *  Icatjl 

^  0  otherwise 

where  IcatjJ  end  Icatjl  aca  tha  number  of  segments 
classified  a a  catagory  i  and  category  j, 
reapactivaly .  Because  all  such  pradicatas  ara  not 
daflnad  batwaan  all  possible  pairs  of  segments, 
tha  normalizing  factor  (tha  denominator)  is 
subject  to  redefinition  on  a  case  by  case  basis. 
Alternatives  to  (2)  ara  described  following  the 
description  of  daetvad  predicates  below. 

An  example  of  a  fuzzy  predicate  P^j^9)  from  our 
domain  ia  tha  ralati  nship  "is  near",  where 
category  1  "is  n  a  <  •'  category  j.  The 

corresponding  derived  predicate  predj_  j  '9>  (k,  l) 
describes  tha  relationship  batwaan  two  segments, 
Sy  in  category  1  and  Sj^  in  catagory  j.  The  sura  of 
pradjj  *9)  (k,  1)  for  all  possible  pairs  of  segments 
ek  and  s^  ia  normalized  by  the  maximum  possible. 

Definitions  of  pradj^ j  *9)  (k,l)  ara  dependent  on  the 
underlying  semantics  of  tha  problem  domain.  One 
approach  ia  to  define  them  propositionally  as 
(0,1)  if  a  measurable  relationship  between  sk  and 
is  within  or  beyond  soma  threshold.  A  second 
approach  preferred  hare  la  to  define  them  as  fuzzy 
truth  functions  on  the  interval  (0,1).  Inverse 
exponential  truth  functions  ara  commonly  used  in 
furry  set  theory  to  measure  the  "nearness"  of  two 
concepts.  An  alternative  nearness  measures  are  in 
(ZI35)  .  For  example,  if  tha  description  of  P^.,'9) 
contains  a  nominal  value  (e.g.,  tha  Stf*  is 
approximately  50  kilometers  from  tha  NW)  then  let 
XQ  represent  tha  nominal  value  and 

prad^(k,l)  ■*  a  "P|Xo"x|  (3) 

where 

X  ia  the  observed  value  corresponding  to  the 
same  measure  (distance,  curvativa,  angle  of 
declination)  batwaan  ak  and  s^ 

P  la  a  constant  contrast  factor  in  [0,1] 
which  emphasizes  the  magnitude  of  tha 
difference  between  tha  observed  and  nominal 
value  whan  increased 

There  are  many  situations  for  which  the  nearness 
measure  ia  not  bounded  by  an  ideal  but  tha  closer 
to  ak  tha  better.  In  such  cases,  XQ  can  be 
replaced  by  zero  in  formula  (3) . 

"Hot  near"  or  "ae  distant  as  possible"  may  be 
measured  by  the  fuzzy  complement  of  (3) . 


f  1  if  sk  and  s^  are  so  related 
pred  (k,  1)  =  4  (5) 

l,  0  otherwise 

If  there  is  a  measure  X  associated  with  the 
relationship  and  Xk  >  X ^  when  the  condition  is 
mat,  the  derived  predicate  of  formula  (5)  can  be 
represented  by  tha  ceiling  function 

-  r  (Xfc-OCjWUJCfc-Xil+l)!  (6) 

For  each  object  attached  to  cat^  requires 

Icatjl  evaluations  of  pred *9)^..  The  multiple 
evaluations  are  combined  to  a  single  value  using 
fuzzy  OR 


max 

»1 

This  corresponds  to  finding  the  best  segment,  s^, 
that  matches  the  relationship  for  a  given  segment 
ak.  By  contract,  the  combination  rule 

( 

min  (pra 
“1 

corresponds  to  fuzzy  AND.  The  heuristic  implied 
by  the  formula  (2)  is 


1’ 


(k, 1) ] ;  for  each  sk  in  c*t± 


(8) 


(prel  ^  (k, 1)J;  for  each  sk  in  catj 


(7) 


Z 

»1 


for  each  sk  in  catj  (9) 


which  corresponds  to  the  average  truth  functional 
value  of  sk  with  all  Sj^  segments  in  catj. 

hot  f<9>1j(k)  stand  for  the  segment  level 
combination  rule,  (7) ,  (8) ,  or  (9) .  Possible 

aggregation  rules  to  compute  rj^9)  are 


I 

•k 


max 

s 

k 


min 

a 

k 


i9’ jkj/lcatil 


(^9) <*)] 


if 

I 

4" 


[J3! (k) J 


(10) 

(11) 

(12) 


which  correspond  to  average,  best,  and  worst 
match,  respectively.  The  aggregation  rule  of 
formula  (10)  Is  the  one  implied  by  formula  (2) . 


prei9>(k,l)  »  1  -  f()  (4) 

where  f ()  is  the  right  side  of  formula  (3) . 

Some  relationships  such  as  "above"  or  "smaller" 
are  not  easily  modelled  as  nearness 


EXAMPLE 

rig.  3  is  a  reproduction  of  Fig.  1(«)  with  most 
segments  labelled  (correctly) .  Eight  segments  are 
labelled  as  Sj,  Sj,  ...,  3g  and  are  used  below  in 
an  example.  Table  1  lists  and  defines  all 
predicates  and  derived  predicate*  required  f'r  th» 
semantic  net  of  rig.  2.  Tha  notation  |cathl 
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refers  to  the  number  of  segments  that  are  an 
instance  of  category  h.  The  value  0.5  is  chosen 
arbitrarily  for  P  in  all  derived  predicates  .  The 
exponential  form  of  derived  predicates  is  used  for 
all  relationships  except  "north  of"  where  formula 
(6)  is  substituted.  The  default  value  for  any 
predicate  or  derived  predicate  is  zero  should  a 
denominator  evaluate  to  zero . 

The  eight  segments  distinguished  in  Fig.  3  are 
characterized  in  Table  2.  For  this  example,  we 
need  only  the  geophysical  coordinates,  the 
distances  between  segment  centroids,  and  the 
distances  between  the  closest  points  of  segments . 
A  larger,  more  complete  description  might  also 
contain  the  length  and  degree  of  curvature  of  each 
segment . 

Table  3  lists  aix  organisms  together  with  their 
fitness  function  values  which  are  computed  using 
the  predicates  in  Table  1.  The  fitness  function 
is  given  by  formula  (2) .  The  combination  and 
aggregation  rules  are  formulas  (7)  and  (12), 
respectively.  Derived  predicates  are  variatlona 


of  formulas  (3)  and  (4)  except  "north  of",  which 
ia  represented  by  formula  (6)  with  the  requisite 
measure  being  latitude.  Organism  Oj  has  no 
segments  labelled  incorrectly.  C>2  has  two 
segments  labelled  incorrectly.  O3  "hrough  Og  have 
3,  3,  5,  and  8  incorrectly  labelled  segments, 
respectively.  The  fitness  function  values 
correspond  roughly  to  the  correctness  of  the 
labelling.  Additional  predicates  (i.e.,  a  more 
complex  semantic  net)  would  improve  upon  the 
ordering  and  separation  in  most  cases. 

CONCLUSION 

A  model  for  labelling  complex  scenes  via  genetic 
algorithms  with  fuzzy  fitness  functions  evaluated 
over  semantic  nets  and  GAs  is  possible.  Truth 
functionals  indicating  the  degree  to  which 
specific  interfeature  relationships  are  fulfilled 
are  combined  at  the  segment  level  then  aggregated 
at  the  category  level  using  fuzzy  set  operators. 

We  are  currently  investigating  such  issues  as  the 
effect  of  many  predicates  clustered  on  one  or  two 
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Figure  3.  Segmented  Image  With  Correct  Labels 


Table  2 .  Segment  Descriptors 


Cantroid  Position  in  fraction#  of  Latitoda  and  Longitud* 

Stqaant  Latltod*  Lonqltuda 

ax  39.49  70.04 

3}  31 . 82  41.49  S1 

39. 33  it. 14  >3 

3,  39.37  49.97  *J 

3,  37.33  99.72  *4 

3,  37.32  99.09  s3 

S7  34.07  93.43  »4 

3(  39.34  94.44  *7 

*4 


b.  Dlitaacaa  Eatnan  Cantrolda  (kllcnaataia) 

*j  4j  Sj  3,  >,  >4  *7  *4 
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29 

39 
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»« 
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- 
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- 
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Table  3.  Fitness  Function  Values  for 
Selected  Organisms 


[YA75] 


°1 

- 

<2JW  SW  NW 

SW  CS  CE  CZ  CZ> 

a (0^,  -  2 .1094 

°2 

<SV  S*-#  NW 

HK  CJ  Ol  CS  CE*> 

KOj)  ■  2.2511 

[ZA88] 

°3 

m 

<NW  SW  KM 

NW  CB  C3  HW  SW>  ; 

*<03}  -  2.1251 

[  Z 1 8  5 1 

°4 

■ 

<3W  SW  NW 

CE  KM  CE  CE  CS> 

I(04)  -  1.4731 

°5 

m 

<NW  NW  CE 

CE  SW  KM  SW  CZ> 

;  *<Os)  »  1.6757 

m 

<SW  CE  SW 

CE  SW  NW  SW  HW> 

;  *<Os)  -  0.9233 
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categories,  alternate  forms  for  the  truth 
functionals  themselves,  and  the  crossover  rules. 
Our  Image  set  consists  of  six  segmented  infrared 
photographs  of  the  North  Atlantic,  each  photograph 
having  a  different  degree  of  observation.  Our 
testbed  will  consist  of  a  GA  algorithm  capable  of 
manipulating  the  alleles'  correspondence  to  the 
semantic  net. 
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